Readers as research detectives

Flaws in research papers are common but it may require arduous detective work to unravel them. Checklists are helpful, but many inconsistencies will only be revealed through repeated cross-checks of every little detail, just like in a crime case. As a major deterrent for dishonesty, raw data from all trials should be posted on a public website. This would also make it much easier to detect errors and flaws in publications, and it would allow many research projects to be performed without collecting new data. The prevailing culture of secrecy and ownership to data is not in the best interests of patients

Flaws in design, analysis and interpretation of Pfizer's antifungal trials of voriconazole and uncritical subsequent quotations

We have previously described how a series of trials sponsored by Pfizer of its antifungal drug, fluconazole, in cancer patients with neutropenia handicapped the control drug, amphotericin B, by flaws in design and analysis. We describe similar problems in two pivotal trials of Pfizer's new antifungal agent, voriconazole, published in a prestigious journal. In a non-inferiority trial, voriconazole was significantly inferior to liposomal amphothericin B, but the authors concluded that voriconazole was a suitable alternative. The second trial used amphothericin B deoxycholate as comparator, but handicapped the drug by not requiring pre-medication to reduce infusion-related toxicity or substitution with electrolytes and fluid to reduce nephrotoxicity, although the planned duration of treatment was 84 days. Voriconazole was given for 77 days on average, but the comparator for only 10 days, which precludes a meaningful comparison. In a random sample of 50 references to these trials, we found that the unwarranted conclusions were mostly uncritically propagated. It was particularly surprising that relevant criticism raised by the FDA related to the first trial was only quoted once, and that none of the articles noted the obvious flaws in the design of the second trial. We suggest that editors ensure that the abstract reflects fairly on the remainder of the paper, and that journals do not impose any time limit for accepting letters that point out serious weaknesses in a study that have not been noted before

Are benefits and harms in mammography screening given equal attention in scientific articles? A cross-sectional study

<p>Abstract</p> <p>Background</p> <p>The CONSORT statement specifies the need for a balanced presentation of both benefits and harms of medical interventions in trial reports. However, invitations to screening and newspaper articles often emphasize benefits and downplay or omit harms, and it is known that scientific articles can be influenced by conflicts of interest. We wanted to determine if a similar imbalance occurs in scientific articles on mammography screening and if it is related to author affiliation.</p> <p>Methods</p> <p>We searched PubMed in April 2005 for articles on mammography screening that mentioned a benefit or a harm and that were published in 2004 in English. Data extraction was performed by three independent investigators, two unblinded and one blinded for article contents, and author names and affiliation, as appropriate. The extracted data were compared and discrepancies resolved by two investigators in a combined analysis. We defined three groups of authors: (1) authors in specialties unrelated to mammography screening, (2) authors in screening-affiliated specialties (radiology or breast cancer surgery) who were not working with screening, or authors funded by cancer charities, and (3) authors (at least one) working directly with mammography screening programmes. We used a data extraction sheet with 17 items described as important benefits and harms in the 2002 WHO/IARC-report on breast cancer screening.</p> <p>Results</p> <p>We identified 854 articles, and 143 were eligible for the study. Most were original research. Benefits were mentioned more often than harms (96% vs 62%, P < 0.001). Fifty-five (38%) articles mentioned only benefits, whereas seven (5%) mentioned only harms (P < 0.001). Overdiagnosis was mentioned in 35 articles (24%), but was more often downplayed or rejected in articles that had authors working with screening, (6/15; 40%) compared with authors affiliated by specialty or funding (1/6; 17%), or authors unrelated with screening (1/14; 7%) (P = 0.03). Benefits in terms of reduced breast cancer mortality were mentioned in 109 (76%) articles, and was more often provided as a relative risk reduction than an absolute risk reduction, where quantified (45 articles (31%) versus 6 articles (3%) (P < 0.001)).</p> <p>Conclusion</p> <p>Scientific articles tend to emphasize the major benefits of mammography screening over its major harms. This imbalance is related to the authors' affiliation.</p

Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): Explanation and Elaboration

In this explanatory and elaboration document Mattias Egger and colleagues provide the meaning and rationale of each checklist item on the STROBE Statement

Dealing with substantial heterogeneity in Cochrane reviews. Cross-sectional study

<p>Abstract</p> <p>Background</p> <p>Dealing with heterogeneity in meta-analyses is often tricky, and there is only limited advice for authors on what to do. We investigated how authors addressed different degrees of heterogeneity, in particular whether they used a fixed effect model, which assumes that all the included studies are estimating the same true effect, or a random effects model where this is not assumed.</p> <p>Methods</p> <p>We sampled randomly 60 Cochrane reviews from 2008, which presented a result in its first meta-analysis with substantial heterogeneity (I²greater than 50%, i.e. more than 50% of the variation is due to heterogeneity rather than chance). We extracted information on choice of statistical model, how the authors had handled the heterogeneity, and assessed the methodological quality of the reviews in relation to this.</p> <p>Results</p> <p>The distribution of heterogeneity was rather uniform in the whole I²interval, 50-100%. A fixed effect model was used in 33 reviews (55%), but there was no correlation between I²and choice of model (P = 0.79). We considered that 20 reviews (33%), 16 of which had used a fixed effect model, had major problems. The most common problems were: use of a fixed effect model and lack of rationale for choice of that model, lack of comment on even severe heterogeneity and of reservations and explanations of its likely causes. The problematic reviews had significantly fewer included trials than other reviews (4.3 vs. 8.0, P = 0.024). The problems became less pronounced with time, as those reviews that were most recently updated more often used a random effects model.</p> <p>Conclusion</p> <p>One-third of Cochrane reviews with substantial heterogeneity had major problems in relation to their handling of heterogeneity. More attention is needed to this issue, as the problems we identified can be essential for the conclusions of the reviews.</p

Divine intervention? A Cochrane review on intercessory prayer gone beyond science and reason

We discuss in this commentary a recent Cochrane review of 10 randomised trials aimed at testing the religious belief that praying to a god can help those who are prayed for. The review concluded that the available studies merit additional research. However, the review presented a scientifically unsound mixture of theological and scientific arguments, and two of the included trials that had a large impact on the findings had problems that were not described in the review. The review fails to live up to the high standards required for Cochrane reviews

Comments on John D. Keen and James E. Keen, What is the point: will screening mammography save my life? BMC Medical Informatics and Decision Making, 2009

This paper by John D. Keen and James E. Keen addresses a thorny subject. The numerical findings and commentaries in their paper will be disturbing to some readers and seem to defy logic and well established viewpoints. It may well generate angry letters to the editor. However such numerical analysis and reporting including civil discussion should be welcomed and are the basis for informed decision making – something that is highly needed in this field

Overdiagnosis in breast cancer screening: the importance of length of observation period and lead time

PMCID: PMC3706885This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited

Patient and public involvement in reducing health and care research waste

Background Eighty five per cent of health research expenditure is potentially wasted due to failure to publish research, unclear reporting of research that is published, and the failure of new research studies to systematically review previous research in the same topic area, poor study design and conduct. A great deal of progress has been made to address this issue but the role of patients and the public has not been considered. Main A small survey was undertaken, as part of a larger programme of work on reducing health and care waste, to understand the role of patients in reducing research waste. The study showed that patients are interested in this issue particularly in relation to the prioritisation of research and patient and public involvement. Conclusions Patients undertake key roles in the research process including co-applicancy, project management, or as co-researchers. This brings responsibility for ensuring high quality research and value for money. Responsibility for recognition of the potential for wasteful practices is part of the conduct and operation of research studies